Only swimming or by boat. You cannot get anywhere on foot unless you're on the island, which you will not be for more than 30min. So be prepared to spend a lot of time in the sea or on the boat.
Abrolhos